Compacting Deep Neural Networks for Internet of Things: Methods and Applications

Deep neural networks (DNNs) have shown great success in completing complex tasks. However, DNNs inevitably bring high computational cost and storage consumption due to the complexity of hierarchical structures, thereby hindering their wide deployment Internet-of-Things (IoT) devices, which limited capability capacity. Therefore, it is a necessity investigate technologies compact DNNs. Despite tremendous advances compacting DNNs, few surveys summarize compacting-DNNs technologies, especially for IoT applications. Hence, this article presents comprehensive study on technologies. We categorize into three major types: 1) network model compression; 2) knowledge distillation (KD); 3) modification structures. also elaborate diversity these approaches make side-by-side comparisons. Moreover, we discuss applications compacted various outline future directions.